Suppress log spams in gcsio 3.0#38588
Conversation
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request addresses excessive logging output that appeared after upgrading to gcsio 3.0. By programmatically adjusting the logging level for the specific class responsible for the alert, the change prevents log spam while maintaining necessary diagnostic information for severe issues. Highlights
New Features🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a mechanism to suppress log spam from the GCS IO library by setting the log level of GoogleCloudStorageImpl to SEVERE during the first invocation of createGoogleCloudStorage. Feedback suggests renaming the AtomicBoolean for better clarity, using a class reference instead of a hardcoded string for the logger name to improve maintainability, and reconsidering the SEVERE log level to avoid suppressing potentially important diagnostic warnings.
| return RetryDeterminer.SOCKET_ERRORS.shouldRetry(e); | ||
| } | ||
| }; | ||
| private static final AtomicBoolean overwriteLog = new AtomicBoolean(false); |
There was a problem hiding this comment.
The variable name overwriteLog is somewhat ambiguous and could be interpreted as overwriting a log file. Renaming it to something like gcsioLoggerConfigured would more clearly indicate that it tracks whether the logging configuration for the GCS IO library has been applied.
| private static final AtomicBoolean overwriteLog = new AtomicBoolean(false); | |
| private static final AtomicBoolean gcsioLoggerConfigured = new AtomicBoolean(false); |
| if (overwriteLog.compareAndSet(false, true)) { | ||
| java.util.logging.Logger gcsLogger = | ||
| java.util.logging.Logger.getLogger( | ||
| "com.google.cloud.hadoop.gcsio.GoogleCloudStorageImpl"); | ||
| gcsLogger.setLevel(java.util.logging.Level.SEVERE); | ||
| } |
There was a problem hiding this comment.
Using GoogleCloudStorageImpl.class.getName() is preferred over a hardcoded string to ensure type safety and ease of refactoring. Additionally, please note that setting the log level to SEVERE is very aggressive as it suppresses all WARNING and INFO messages from this class, which might contain important diagnostic information. If the intention is only to suppress the specific "ALERT" message, consider if WARNING is sufficient or if this suppression should be made configurable.
if (gcsioLoggerConfigured.compareAndSet(false, true)) {
java.util.logging.Logger gcsLogger =
java.util.logging.Logger.getLogger(GoogleCloudStorageImpl.class.getName());
gcsLogger.setLevel(java.util.logging.Level.SEVERE);
}There was a problem hiding this comment.
The spamming log is a WARN log, see PR description
| GoogleCloudStorage createGoogleCloudStorage( | ||
| GoogleCloudStorageOptions options, Storage storage, Credentials credentials) | ||
| throws IOException { | ||
| // Suppress log spams in gcsio 3.0 |
There was a problem hiding this comment.
Tested working with ./gradlew :sdks:java:testing:jpms-tests:dataflowRunnerIntegrationTest -info. If put it in a static block it's not effective because the root logger hasn't been initialized yet
|
R: @Amar3tto |
|
R: @shunping |
|
Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment |
1 similar comment
|
Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment |
static reference exist in GoogleCloudStorageImpl
2 participants
Please add a meaningful description for your change here
There is a
"ALERT: Overriding httpRequestInitializer - this should not be done in production!"log added in gcsio 3.0. However, Beam does use a custom HttpInitializer handling retry/user-agent/ignore 404 warnings etcThank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, commentfixes #<ISSUE NUMBER>instead.CHANGES.mdwith noteworthy changes.See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)
See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.